Method and apparatus for video encoding and/or decoding to prevent start code confusion

ABSTRACT

A method and a video processor for preventing start code confusion. The method includes aligning bytes of a slice header relating to slice data when the slice header is not byte aligned or inserting differential data at the end of the slice header before the slice data when the slice header is byte aligned, performing emulation prevention byte insertion on the slice header, and combine the slice header and the slice data after performing emulation prevention byte insertion.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and incorporates by reference U.S.Provisional Application No. 61/499,276, filed Jun. 21, 2011 and U.S.Provisional Application No. 61/502,387, filed on Jun. 29, 2011.

FIELD OF THE INVENTION

This invention generally relates to method and apparatus utilized in anencoder and/or decoder to prevent misidentifying a start code of a videobitstream, for example, when the processing order to generate the syntaxelements and the signaling order of the syntax elements within thebitstream are different.

BACKGROUND OF THE INVENTION

FIG. 1 is an embodiment depicting a video processing system. In FIG. 1,a video processing system 100 utilizes an encoder 200 and a decoder 110.The encoder 200 receives video data 120. The encoder 200 encodes in apredetermined format encoded data 130. The decoder 110 receives theencoded data 130 and decodes it according to the predetermined format toproduce decoded data 140. The decoder has to properly identify thebeginning of each bitstream. The encoder utilizes an identifier tohighlight the beginning of the bitstream to the decoder. However,sometimes, the identifier may accidentally exist in the encoded streamwhere it is not the beginning of the stream.

For example, at the encoder, an existing method of emulation byteprevention (EPB) insertion is used to detect this identifier within thestream and replace it with an alternative pattern. At the decoder, oncethe beginning of the stream is detected, EPB removal is performed on therest of the stream. When the syntax elements that are to be encoded intothe bitstream is generated in the same order that they are to besignaled in the bitstream, EPB insertion at the encoder may be done onthe fly as the bitstream is being generated. This enables low latencyencoding, since there is no additional delay of performing EPBinsertion. However, when the syntax elements that is to be encoded intothe bitstream are not generated in the same order that they areasignaled in the bitstream, for instance, when syntax elements generatedfrom step 2 are inserted before syntax elements generated from step 1 inthe bitstream. , EPB cannot be done of the fly without potentiallycausing the identifier to appear at the transition point between bitsfrom step 1 and 2.

Therefore, there is a need for an improved method and apparatus thatprevents the encoder from misleading the decoder and that assists thedecoder to correctly identify beginning of a bitstream

SUMMARY

Embodiment of the current invention relate to a method and a videoprocessor for preventing start code confusion. The method includesaligning bytes of a slice header relating to slice data when the sliceheader is not byte aligned or inserting differential data at the end ofthe slice header before the slice data when the slice header is bytealigned, performing emulation prevention byte insertion on the sliceheader, and combine the slice header and the slice data after performingemulation prevention byte insertion.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the presentinvention can be understood in detail, a more particular description ofthe invention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 is an embodiment depicting a video processing system;

FIG. 2 is an embodiment depicting an encoder;

FIG. 3 is an embodiment of a bitstream structure in High EfficiencyVideo Coding (HEVC) standard;

FIG. 4 is an embodiment of a flow diagram for a method for preventingstart code confusion;

FIG. 5 is another embodiment of a flow diagram for a method forpreventing start code confusion; and

FIG. 6 is an embodiment depicting Emulation Prevention Byte (EPB)insertion problem.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

FIG. 2 is an embodiment depicting an encoder 200. In FIG. 2, the encoder200 includes an intra/inter prediction module 202, a transformquantization module 204, a deblocking module 206, a Sample AdaptiveOffset module (SAO) 208 and an Adaptive Loop Filter (ALF) 210. Videopixels 201 are processed by the encoder 200 to produce a bitstream 300,described in more detail in FIG. 3. Such processing may be block orframe based. The beginning of each slice in a bitstream is highlightedby an identifier, herein called the “start code”. The slice header isvariable in size and usually precedes the slice data.

FIG. 3 is an embodiment of a bitstream 300 structure in High EfficiencyVideo Coding (HEVC) standard. In FIG. 3, the bitstream 300 includes aslice header 302 and slice data (LCU) 304. The slice header 302 mayinclude slice type 306, SAO parameters 308, and ALF coefficients 310.

For example, the SAO and ALF syntax elements are generated after theslice data syntax elements, but inserted in the slice header whichappears before the slice data in the bitstream. Thus, the syntaxelements generation and the signaling order are different. The sliceheader bits are generated after the slice data bits even though itappears before the slice data in the bitstream. EPB insertion may not beapplied to slice data until after bits of the slice header aregenerated, i.e. until after SAO and/or ALF processing is complete todetermine the syntax elements to be encoded into the slice header. As aresult, there may be an increased delay. Unless the slice header isproperly handled, if EPB insertion is done on slice data before sliceheader is generated, the identifier may appear at the transition betweenslice header and slice data bits.

It is noted that when EPB insertion is done on slice data 304 to removethe start code, if slice header is added afterwards in front of theslice data, the start code can appear due to the combination of the endof slice header and the start of the slice data. Hence, a start code mayincorrectly be detected, where a portion of the start code is at the endof the slice header and the rest of the start code is in the slice data.For example, a portion of the start code may be located at the end ofthe slice header (e.g. in the ALF coefficient 310) and the rest of thestart code may be at the beginning of the slice data. If the start codeis ‘000001’, and the slice header ends with ‘00’ and the first LCUstarts with ‘0001’, a decoder may mistakenly decode a start code.

FIG. 4 is an embodiment of a flow diagram for a method 400 forpreventing start code confusion when parameters in the slice header aregenerated after the slice data. The method 400 starts at step 402 andproceeds to step 404. At step 404, the method 400 determines if theslice header is byte aligned, i.e. if the header is in 8 bits multiples.If it is not aligned, the method 400 proceeds to step 408 wherein themethod 400 byte aligns the slice header. In one embodiment, the method400 utilizes Raw Byte Sequence Payload (RBSP) byte alignment to performthe byte alignment. In one embodiment, the header byte alignment isaccomplish by inserting 1's at the end of the slice header. If the sliceheader is byte aligned, i.e. the header is in 8 bits multiples, themethod 400 proceeds to step 406. At step 406, the method 400 insertsdifferentiating data at the end of the slice header, i.e. ones or binary‘FF’. The method 400 proceeds to step 410 from steps 406 and 408.

At step 410, the method 400 performs Emulation Prevention Byte Insertionon the slice header. In one embodiment, the emulation prevention byteinsertion is performed on the header in parallel, simultaneously, withperforming the emulation prevention byte insertion on the slice data. Inanother embodiment, the emulation prevention byte insertion is performedon the header after performing emulation prevention byte insertion onthe slice data. At step 412, the method 400 combines the slice headerand the slice data. In one embodiment, the slice header and the slicedata are combined after the Emulation Prevention Byte Insertion isperformed. The method 400 ends at step 414.

By byte aligning the slice header or inserting differentiating data atthe end of the slice header, EPB can be done independently on the sliceheader and the slice data (i.e. EPB insertion in slice header does notaffect slice data. Accordingly, on the fly EPB insertion can beperformed on slice data before slice header is generated. Once combined,the start code will not appear at their transition point, which reducesthe delay of the encoder.

In the case where the start code is ‘000001’ and the slice header isvariable in size, the method 400 byte aligns the slice header, i.e., Ifthere is no byte alignment and the last bits of the slice header arex00, alignment is done by using RBSP or insertion of ones or byinserting differentiating data, such as ‘FF’. Alternatively, FF canalways be inserted at the end of the slice header. The method 400performs EPB insertion to ensure that the start code was notaccidentally created in the header where a start is not intended. In thecase that EPB insertion has already been performed on the slice data,EPB insertion may only be performed on the slice header or a portionthereof.

FIG. 5 is another embodiment of a flow diagram for a method 500 forpreventing start code confusion. The method 500 starts at step 502 andproceeds to step 504. At step 504, the method 500 determines if theslice header is byte aligned, i.e. if the header is in 8 bits multiples.If slice header is not aligned, the method 500 proceeds to step 508wherein the method 500 byte aligns the slice header. In one embodiment,the method 500 utilizes Raw Byte Sequence Payload (RBSP) byte alignmentto byte align the slice header. In another embodiment, the header bytealignment is accomplished by inserting 1's at the end of the sliceheader. If the slice header is byte aligned, i.e. the header is in 8bits multiples, the method 500 proceeds to step 506. At step 506, themethod 500 inserts differentiating data at the end of the slice header,i.e. the differentiating data may be ones or a binary ‘FF’. The method500 proceeds to step 510 from steps 506 and 508.

At step 510, the method 500 determines if there is a potential for codeconfusion. If there exists start code confusion (i.e., the slice headerends with zeros), the method 500 proceeds to step 512, wherein insertsdifferentiating data at the end of the slice header, i.e., ones or abinary ‘FF’. At step 514, the method 500 performs EPB insertion on theslice header and the method 500 proceeds to step 516. In one embodiment,the emulation prevention byte insertion is performed on the header inparallel, simultaneously, with performing the emulation prevention byteinsertion on the slice data. In another embodiment, the emulationprevention byte insertion is performed on the header after performingemulation prevention byte insertion on the slice data. If there is nopotential start code confusion, the method 500 proceeds to step 516. Themethod 500 ends at step 516.

FIG. 6 is an embodiment depicting Emulation Prevention Byte (EPB)insertion problem. EPB, for example 0×03, may be inserted into thebitstream (i.e. RBSP data) after entropy coding to ensure that forconsecutive byte-aligned bits, there are no repetitions of anyspecial/reserved codes, i.e. start code prefix ‘0x000001’. In H.264/AVCEPB is defined as: ‘00000000 00000000 000000xx’ (where xx represents any2 bit pattern: 00, 01, 10, or 11), and a byte equal to 0×03 is insertedto replace these bit patterns with the patterns: ‘00000000 0000000000000011 000000xx’.”

The EPB insertion may be done immediately after entropy coding of slicedata. However, inserting the SAO parameters and ALF coefficients in theslice header may affect EPB insertion on the slice data bits, forexample, slice header may not be byte aligned. In such a case, toprevent change in byte alignment, it is preferred that EPB is not donefor slice dataIn one embodiment, EPB insertion is performed on theentire bitstream after SAO and ALF processes have completed. Thisfurther increases the delay which is not suitable for low delayapplications such as video conferencing. In addition, in CABAC, whenentropy coding is performed on SAO parameters and ALF coefficients, theentire sequence may not be fully entropy coded (i.e. compress binarysymbols (bins) to bits) until the SAO parameters and ALF coefficientsare determined.

In such embodiments, EPB may not be performed on the slice datasimultaneously as the SAO and/or the ALF processes are taking place.Once complete, EPB may be performed on the bistream after the SAO and/orALF are completed. To enable EPB to be performed on the slice datasimultaneously, as the SAO and/or the ALF processes are taking place,the slice header is byte aligned.

In one embodiment, the byte alignment is done with ones. If no bytealignment is needed, but the slice header ends with x00, an additionalxFF may be inserted. For simplicity, rather than checking the last byteof the slice header, xFF may always be inserted when byte alignment isnot needed. In another embodiment, RBSP byte alignment may be utilized,where a stop bit of value 1 is always inserted and the rest of the bytealignment bits can be zero. The slice header and slice data can bestitched together once the slice header is processing is complete. As aresult, EPB insertion (or removal at decoder) may be doneindependently/decoupled from the entropy coding. Such an approach may beimplemented to any syntax elements related to processing that occurs atthe end of the encoding process, for example, after slice data syntaxelements are generated and where the syntax elements generated at theend of the encoding process are inserted before the slice data.

In one embodiment, EPB insertion is performed on slice data before or inparallel with ALF and SAO. When ALF and SAO are completed, EPB insertionis performed up until the end of the slice header, i.e. for the lastallowable EPB insertion, the three-byte pattern it replaces, x0000xx,may include bits from the slice header. At the decoder, EPB removal isperformed until slice header is reached, i.e. the last allowable EPBremoval, the three-byte pattern, x000003, may include bits from theslice header. Thus, in one embodiment, EPB insertion/removal may requireknowledge of the slice header/slice data boundary and/or the locationwhere ALF coefficients end.

In another embodiment, the SAO parameters and/or ALF coefficients areplaced in a separate NALU. These NALU can be transmitted together in aslice or separately, i.e. the NALU of the slice data can be transmittedfirst for low latency applications. Such an approach may be implementedto any syntax elements related to processing that occurs at the end ofthe encoding process, after slice data syntax elements are generated,and where the syntax elements generated at the end of the encodingprocess are inserted before the slice data.

In one embodiment, CABAC termination and byte alignment of the sliceheader are only performed when a syntax element related to end ofencoding process is inserted in the slice header.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

What is claimed is:
 1. A method of a video processor for preventingstart code confusion, the method comprising: aligning bytes of a sliceheader relating to slice data when the slice header is not byte alignedor inserting differential data at the end of the slice header before theslice data when the slice header is byte aligned; performing emulationprevention byte insertion on the slice header; and combine the sliceheader and the slice data after performing emulation prevention byteinsertion.
 2. The method of claim 1, wherein the emulation preventionbyte insertion is performed on the slice header and slice data.
 3. Themethod of claim 2, wherein the emulation prevention byte insertion isperformed on the header at least one of in parallel or after emulationprevention byte insertion is performed on the slice data.
 4. The methodof claim 1, wherein the byte aligning is accomplished by at least one ofperforming Raw Byte Sequence Payload byte alignment on the slice headeror by inserting 1's at the end of the slice header.
 5. The method ofclaim 1, wherein the inserting of differential data only happens whenthere is potential start code confusion.
 6. The method of claim 5,wherein the confusion is possible when the slice header ends with zeros.7. The method of claim 1, wherein the differentiating code is a binary‘FF’.
 8. A video processor for preventing start code confusion,comprising: means for aligning bytes of a slice header relating to slicedata when the slice header is not byte aligned or inserting differentialdata at the end of the slice header before the slice data when the sliceheader is byte aligned; means for performing emulation prevention byteinsertion on the slice header; and means for combine the slice headerand the slice data after performing emulation prevention byte insertion.9. The video processor of claim 8, wherein the emulation prevention byteinsertion is performed on the slice header and slice data.
 10. The videoprocessor of claim 9, wherein the emulation prevention byte insertion isperformed on the header at least one of in parallel or after emulationprevention byte insertion is performed on the slice data.
 11. The videoprocessor of claim 8, wherein the byte aligning is accomplished by atleast one of performing Raw Byte Sequence Payload byte alignment on theslice header or by inserting 1's at the end of the slice header.
 12. Thevideo processor of claim 8, wherein the inserting of differential dataonly happens when there is potential start code confusion.
 13. The videoprocessor of claim 12, wherein the confusion is possible when the sliceheader ends with zeros.
 14. The video processor of claim 8, wherein thedifferentiating code is a binary ‘FF’.
 15. A non-transitory computerreadable medium with computer instruction, when executed, the computerinstructions perform a method of a video processor for preventing startcode confusion, the method comprising: aligning bytes of a slice headerrelating to slice data when the slice header is not byte aligned orinserting differential data at the end of the slice header before theslice data when the slice header is byte aligned; performing emulationprevention byte insertion on the slice header; and combine the sliceheader and the slice data after performing emulation prevention byteinsertion.
 16. The non-transitory computer readable medium of claim 15,wherein the emulation prevention byte insertion is performed on theslice header and slice data.
 17. The non-transitory computer readablemedium of claim 16, wherein the emulation prevention byte insertion isperformed on the header at least one of in parallel or after emulationprevention byte insertion is performed on the slice data.
 18. Thenon-transitory computer readable medium of claim 15, wherein the bytealigning is accomplished by at least one of performing Raw Byte SequencePayload byte alignment on the slice header or by inserting 1's at theend of the slice header.
 19. The non-transitory computer readable mediumof claim 15, wherein the inserting of differential data only happenswhen there is potential start code confusion.
 20. The non-transitorycomputer readable medium of claim 19, wherein the confusion is possiblewhen the slice header ends with zeros.
 21. The non-transitory computerreadable medium of claim 15, wherein the differentiating code is abinary ‘FF’.